Feature fusion within local region using localized maximum-margin learning for scene categorization

In the field of visual recognition such as scene categorization, representing an image based on the local feature (e.g. the bags of visual words (BOW) model and its variants) has become popular and one of the most successful methods. In this paper, we propose a method that uses localized maximum-margin learning to fuse different types of features during the BOW modeling process for eventual scene classification. Unlike previous feature fusion methods for visual recognition, which combines the features after generating the entire set of representations based on different types of features from local regions, the proposed method fuses different features at the stage when the best visual word is selected to represent a local region (hard assignment) or the probabilities of the candidate visual words used to represent the unknown region are estimated (soft assignment). The merits of the proposed method are that (1) errors caused by the ambiguity of single feature when assigning local regions to the best representative visual words can be corrected or the probabilities of the candidate visual words used to represent the region can be estimated more accurately; and that (2) it offers a more flexible way in fusing these features through determining the similarity-metric locally by localized maximum-margin learning. The proposed method has been evaluated experimentally and the results indicate its effectiveness.